1. Field of the Invention
The present invention relates to an image processing apparatus, and more particularly to an interpolation process for sampled image signals.
2. Description of the Related Art
Digital still cameras and video cameras have become quite popular among consumers. These cameras typically have image pick up devices that employ a color filter system known as the primary-color Bayer array.
As shown in FIG. 24, the Bayer array is patterned such that each pixel has an R, G, or B color value. To obtain other color values for each pixel, an interpolation process using nearby pixels having the desired color value is required. For example, a G value can be obtained for a B pixel by interpolating G pixels around the B pixel. Such an interpolation process is generally called a simultaneous interpolation process.
When simultaneous interpolation of G signals is performed, a frequency spectrum shown in FIG. 25 is produced because the G signals are subjected to offset sampling as shown in FIG. 27.
For these G signals, aliasing occurs at the sampling frequency positions (indicated by a mark ●). A reproducible frequency domain is then produced. This frequency domain is a rhombic-shaped region, the periphery of which is defined by Nyquist frequency positions (indicated by a mark ◯).
Accordingly, when the simultaneous interpolation process is generally performed by cascading vertical and horizontal low-pass filters, interpolation optimum for such a rhombic frequency domain cannot be achieved, thus resulting in a deterioration of image quality.
More specifically, when the interpolation is uniformly performed by using an interpolation filter that allows passage of signals in bands lower than a vertical Nyquist frequency Vnq and a horizontal Nyquist frequency Hnq. As shown in FIG. 26A, image quality near points Dnq deteriorates due to turn-back signals about points Dfs.
Also, when the interpolation is uniformly performed by using an interpolation filter that allows passage of signals in bands within a quadrilateral region, which is defined by connecting the points Dnq, as shown in FIG. 26B, higher-band signals near the horizontal Nyquist frequency Hnq and the vertical Nyquist frequency Vnq disappear, thus resulting in a blurred image.
To eliminate those adverse effects on image quality caused when uniformly interpolating the entirety of an image by an interpolation filter having a single characteristic, U.S. Pat. No. 5,382,976 ('976 patent) and Japanese Patent Laid-Open No. 10-150668 ('668 patent) (corresponding to U.S. Pat. No. 6,295,087) propose an adaptive simultaneous interpolation process in which correlation between a target pixel and nearby pixels is detected for each pixel and an optimum interpolation method is changed over depending on the detected correlation.
According to the '976 patent, horizontal and vertical high-frequency components are detected from an input image signal. When the input image signal contains high-frequency components in the horizontal direction at a large proportion, it is determined as a vertically striped signal having stronger correlation in the vertical direction, and an interpolation process suitable for the vertically striped signal is performed. When the input image signal contains high-frequency components in the vertical direction at a large proportion, it is determined as a horizontally striped signal having stronger correlation in the horizontal direction, and an interpolation process suitable for the horizontally striped signal is performed. When those two conditions are not satisfied, the interpolation process is performed using an average value among surrounding pixels.
Also, according to Japanese Patent Laid-Open No. 10-150668, degrees of correlation in horizontal, vertical and diagonal directions are computed and the interpolation process is performed in one of those three directions in which the correlation degree is maximum.
However, the related art is disadvantageous because a satisfactory interpolation process based on the correlation cannot be performed for pixels having correlation in directions other than the vertical and horizontal directions, and resolution in appearance deteriorates in the diagonal direction.
Further, in the adaptive interpolation process disclosed in the '976 patent, to determine a direction in which a preferable interpolation process is to be performed, high-frequency components in the vertical and horizontal directions contained in the image signal are first determined by computation. This process for detecting a correlation value has a band-pass characteristic such that a signal with a frequency component that is ¼ of the image sampling frequency fs reaches its peak, and signals in bands higher and lower than the ¼ frequency attenuate bilaterally symmetrically on both sides of fs/4.
When that correlation value extracting process is applied to signals in regions near the vertical and horizontal Nyquist frequencies, the correlation values in the vertical and horizontal directions should be ideally output as equal values because each signal in those regions contains a high-band component close to the Nyquist frequency in one of the vertical and horizontal directions and a low-band component close to the DC component in the other direction.
In practice, however, a difference occurs between the vertical correlation value and the horizontal correlation value due to effects of noise and an optical LPF, and the relation in magnitude between those two correlation values varies per pixel. Therefore, an error may be caused in determining the correlation direction using the vertical correlation value and the horizontal correlation value. This causes the interpolation result to change unnaturally per pixel thus resulting in deterioration of image quality.
According to the '668 patent, the number of directions for detecting the correlation is increased to more finely control a region in which a preferable interpolation process is to be performed. However, the interpolation result is changed discontinuously each time the determination result of the correlation direction changes, thus resulting in unnaturalness in image quality.